Mathematics of biomedical imaging today—a perspective

Biomedical imaging is a fascinating, rich and dynamic research area, which has huge importance in biomedical research and clinical practice alike. The key technology behind the processing, and automated analysis and quantification of imaging data is mathematics. Starting with the optimisation of the image acquisition and the reconstruction of an image from indirect tomographic measurement data, all the way to the automated segmentation of tumours in medical images and the design of optimal treatment plans based on image biomarkers, mathematics appears in all of these in different flavours. Non-smooth optimisation in the context of sparsity-promoting image priors, partial differential equations for image registration and motion estimation, and deep neural networks for image segmentation, to name just a few. In this article, we present and review mathematical topics that arise within the whole biomedical imaging pipeline, from tomographic measurements to clinical support tools, and highlight some modern topics and open problems. The article is addressed to both biomedical researchers who want to get a taste of where mathematics arises in biomedical imaging as well as mathematicians who are interested in what mathematical challenges biomedical imaging research entails

Nonparametric image registration of airborne LiDAR, hyperspectral and photographic imagery of wooded landscapes

There is much current interest in using multisensor airborne remote sensing to monitor the structure and biodiversity of woodlands. This paper addresses the application of nonparametric (NP) image-registration techniques to precisely align images obtained from multisensor imaging, which is critical for the successful identification of individual trees using object recognition approaches. NP image registration, in particular, the technique of optimizing an objective function, containing similarity and regularization terms, provides a flexible approach for image registration. Here, we develop a NP registration approach, in which a normalized gradient field is used to quantify similarity, and curvature is used for regularization (NGF-Curv method). Using a survey of woodlands in southern Spain as an example, we show that NGF-Curv can be successful at fusing data sets when there is little prior knowledge about how the data sets are interrelated (i.e., in the absence of ground control points). The validity of NGF-Curv in airborne remote sensing is demonstrated by a series of experiments. We show that NGF-Curv is capable of aligning images precisely, making it a valuable component of algorithms designed to identify objects, such as trees, within multisensor data sets.This work was supported by the Airborne Research and Survey Facility of the U.K.’s Natural Environment Research Council (NERC) for collecting and preprocessing the data used in this research project [EU11/03/100], and by the grants supported from King Abdullah University of Science Technology and Wellcome Trust (BBSRC). D. Coomes was supported by a grant from NERC (NE/K016377/1) and funding from DEFRA and the BBSRC to develop methods for monitoring ash dieback from aircraft.This is the final version. It was first published by IEEE at http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7116541&sortType%3Dasc_p_Sequence%26filter%3DAND%28p_Publication_Number%3A36%29%26pageNumber%3D5

Adversarial regularizers in inverse problems

Inverse Problems in medical imaging and computer vision are traditionally solved using purely model-based methods. Among those variational regularization models are one of the most popular approaches. We propose a new framework for applying data-driven approaches to inverse problems, using a neural network as a regularization functional. The network learns to discriminate between the distribution of ground truth images and the distribution of unregularized reconstructions. Once trained, the network is applied to the inverse problem by solving the corresponding variational problem. Unlike other data-based approaches for inverse problems, the algorithm can be applied even if only unsupervised training data is available. Experiments demonstrate the potential of the framework for denoising on the BSDS dataset and for computed tomography reconstruction on the LIDC dataset.The authors acknowledge the National Cancer Institute and the Foundation for the National Institutes of Health, and their critical role in the creation of the free publicly available LIDC/IDRI Database used in this study. The work by Sebastian Lunz was supported by the EPSRC grant EP/L016516/1 for the University of Cambridge Centre for Doctoral Training, the Cambridge Centre for Analysis and by the Cantab Capital Institute for the Mathematics of Information. The work by Ozan Öktem was supported by the Swedish Foundation for Strategic Research grant AM13-0049. Carola-Bibiane Schönlieb acknowledges support from the Leverhulme Trust project on ‘Breaking the non-convexity barrier’, EPSRC grant Nr. EP/M00483X/1, the EPSRC Centre Nr. EP/N014588/1, the RISE projects CHiPS and NoMADS, the Cantab Capital Institute for the Mathematics of Information and the Alan Turing Institute

Anisotropic osmosis filtering for shadow removal in images

We present an anisotropic extension of the isotropic osmosis model that has been introduced by Weickert et al.~(Weickert, 2013) for visual computing applications, and we adapt it specifically to shadow removal applications. We show that in the integrable setting, linear anisotropic osmosis minimises an energy that involves a suitable quadratic form which models local directional structures. In our shadow removal applications we estimate the local structure via a modified tensor voting approach (Moreno, 2012) and use this information within an anisotropic diffusion inpainting that resembles edge-enhancing anisotropic diffusion inpainting (Weickert, 2006, Gali\'c, 2008). Our numerical scheme combines the nonnegativity preserving stencil of Fehrenbach and Mirebeau (Fehrenbach, 2014) with an exact time stepping based on highly accurate polynomial approximations of the matrix exponential. The resulting anisotropic model is tested on several synthetic and natural images corrupted by constant shadows. We show that it outperforms isotropic osmosis, since it does not suffer from blurring artefacts at the shadow boundaries

Deep learning as optimal control problems: Models and numerical methods

We consider recent work of Haber and Ruthotto 2017 and Chang et al. 2018, where deep learning neural networks have been interpreted as discretisations of an optimal control problem subject to an ordinary differential equation constraint. We review the first order conditions for optimality, and the conditions ensuring optimality after discretisation. This leads to a class of algorithms for solving the discrete optimal control problem which guarantee that the corresponding discrete necessary conditions for optimality are fulfilled. The differential equation setting lends itself to learning additional parameters such as the time discretisation. We explore this extension alongside natural constraints (e.g. time steps lie in a simplex). We compare these deep learning algorithms numerically in terms of induced flow and generalisation ability

Guidefill: GPU accelerated, artist guided geometric inpainting for 3D conversion of film

The conversion of traditional film into stereo 3D has become an important problem in the past decade. One of the main bottlenecks is a disocclusion step, which in commercial 3D conversion is usually done by teams of artists armed with a toolbox of inpainting algorithms. A current difficulty in this is that most available algorithms either are too slow for interactive use or provide no intuitive means for users to tweak the output. In this paper we present a new fast inpainting algorithm based on transporting along automatically detected splines, which the user may edit. Our algorithm is implemented on the GPU and fills the inpainting domain in successive shells that adapt their shape on the y. In order to allocate GPU resources as efficiently as possible, we propose a parallel algorithm to track the inpainting interface as it evolves, ensuring that no resources are wasted on pixels that are not currently being worked on. Theoretical analyses of the time and processor complexity of our algorithm without and with tracking (as well as numerous numerical experiments) demonstrate the merits of the latter. Our transport mechanism is similar to the one used in coherence transport [F. Bornemann and T. März, J. Math. Imaging Vision, 28 (2007), pp. 259-278; T. März, SIAM J. Imaging Sci., 4 (2011), pp. 981-1000] but improves upon it by correcting a \kinking" phenomenon whereby extrapolated isophotes may bend at the boundary of the inpainting domain. Theoretical results explaining this phenomenon and its resolution are presented. Although our method ignores texture, in many cases this is not a problem due to the thin inpainting domains in 3D conversion. Experimental results show that our method can achieve a visual quality that is competitive with the state of the art while maintaining interactive speeds and providing the user with an intuitive interface to tweak the results.The work of the first author was supported by the Cambridge Commonwealth Trust and the Cambridge Center for Analysis. The work of the third author was supported by the Leverhulme Trust project Breaking the Nonconvexity Barrier, the EPSRC grants EP/M00483X/1 and EP/N014588/1, the Cantab Capital Institute for the Mathematics of Information, the CHiPS (Horizon 2020 RISE project grant), the Global Alliance project “Statistical and Mathematical Theory of Imaging,” and the Alan Turing Institute

Graph Clustering, Variational Image Segmentation Methods and Hough Transform Scale Detection for Object Measurement in Images

© 2016, Springer Science+Business Media New York. We consider the problem of scale detection in images where a region of interest is present together with a measurement tool (e.g. a ruler). For the segmentation part, we focus on the graph-based method presented in Bertozzi and Flenner (Multiscale Model Simul 10(3):1090–1118, 2012) which reinterprets classical continuous Ginzburg–Landau minimisation models in a totally discrete framework. To overcome the numerical difficulties due to the large size of the images considered, we use matrix completion and splitting techniques. The scale on the measurement tool is detected via a Hough transform-based algorithm. The method is then applied to some measurement tasks arising in real-world applications such as zoology, medicine and archaeology

Accelerating variance-reduced stochastic gradient methods

Funder: Gates Cambridge Trust (GB)AbstractVariance reduction is a crucial tool for improving the slow convergence of stochastic gradient descent. Only a few variance-reduced methods, however, have yet been shown to directly benefit from Nesterov’s acceleration techniques to match the convergence rates of accelerated gradient methods. Such approaches rely on “negative momentum”, a technique for further variance reduction that is generally specific to the SVRG gradient estimator. In this work, we show for the first time that negative momentum is unnecessary for acceleration and develop a universal acceleration framework that allows all popular variance-reduced methods to achieve accelerated convergence rates. The constants appearing in these rates, including their dependence on the number of functions n, scale with the mean-squared-error and bias of the gradient estimator. In a series of numerical experiments, we demonstrate that versions of SAGA, SVRG, SARAH, and SARGE using our framework significantly outperform non-accelerated versions and compare favourably with algorithms using negative momentum.</jats:p

Learning optimal spatially-dependent regularization parameters in total variation image denoising

We consider a bilevel optimization approach in function space for the choice of spatially dependent regularization parameters in TV image denoising models. First- and second-order optimality conditions for the bilevel problem are studied when the spatially-dependent parameter belongs to the Sobolev space H1(Ω). A combined Schwarz domain decomposition-semismooth Newton method is proposed for the solution of the full optimality system and local superlinear convergence of the semismooth Newton method is verified. Exhaustive numerical computations are finally carried out to show the suitability of the approach

Directional sinogram inpainting for limited angle tomography

In this paper we propose a new joint model for the reconstruction of tomography data under limited angle sampling regimes. In many applications of Tomography, e.g. Electron Microscopy and Mammography, physical limitations on acquisition lead to regions of data which cannot be sampled. Depending on the severity of the restriction, reconstructions can contain severe, characteristic, artefacts. Our model aims to address these artefacts by inpainting the missing data simultaneously with the reconstruction. Numerically, this problem naturally evolves to require the minimisation of a non-convex and non-smooth functional so we review recent work in this topic and extend results to fit an alternating (block) descent framework. We illustrate the effectiveness of this approach with numerical experiments on two synthetic datasets and one Electron Microscopy dataset.Cantab Capital Institute for the Mathematics of Information PIHC innovation fund of the Technical Medical Centre of UT Dutch 4TU programme Precision Medicine Netherlands Organization for Scientific Research (NWO), project 639.073.506 Henslow Research Fellowship at Girton College, Cambridge Clare College Junior Research Fellowshi